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This letter elucidates the connection between the KS 
entropy-rate k and the time evolution of the physical or sta- 
tistical entropy S. For a large family of chaotic conservative 
dynamical systems including the simplest ones, the evolution 
of S{t) for far-from-equilibrium processes includes a stage dur- 
ing which S is a simple linear function of time whose slope is k. 
The letter presents numerical confirmation of this connection 
for a number of chaotic symplectic maps, ranging from the 
simplest 2-dimensional ones to a 4-dimensional and strongly 
nonlinear map. 
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This paper tries to clarify the connection between the 
Kolmogorov-Sinai entropy and the physical entropy for 
a chaotic conservative dynamical system. This connec- 
tion is obviously very important if one is to understand 
the impact on thermodynamics and statistical mechanics 
of the large amount of work done by mathematicians on 
the behavior of chaotic systems. To start with the KS 
entropy, it is not really an entropy but an entropy per 
unit time, or an "entropy-rate" . ft is a single number 
K, which is a property solely of the chaotic dynamical 
system considered. As for the physical entropy S{t), the 
entropy of the second law of thermodynamics, it is a func- 
tion of time, and this function depends not only on the 
particular dynamical system, but also on the choice of 
an initial probability distribution for the state of that 
system. Though it is clear that the original definition of 
K Q was meant to provide a connection with S{t), the 
precise connection does not seem to be well known nowa- 
days, and the few statements found in the textbooks are 
often vague 1^. 

The simplest connection one might guess would be this: 
the KS entropy-rate would be the maximum possible ab- 
solute value of the rate of variation of the physical en- 
tropy, i.e. \dS/dt\ < K. But this is wrong, because a 
counterexample can easily be found ^j^- The actual 
connection is less direct and, in many cases, it requires 
that S{t) be averaged over many histories (or trajecto- 
ries), so as to give equal weights to initial distributions 
from all regions of phase space. Then, assuming these 
initial distributions to be very far from equilibrium, the 
variation with time of the physical entropy goes through 
three successive, roughly separated stages. In the first 
stage, S{t) is heavily dependent on the details of the dy- 
namical system and of the initial distribution; no generic 



statement can be made; dS/dt can be positive or nega- 
tive, large or small, and in particular it can be larger than 
K. In the second stage, S{t) is a linear increasing function 
whose slope is k. In the third stage, S{t) tends asymp- 
totically toward the constant value which characterizes 
equilibrium, for which the distribution is uniform in the 
available part of phase space. It may happen, however, 
that the simple and generic stage 2 is absent, with stages 
I and 3 merging into each other. This is true when the 
initial distribution is not sufficientely different from the 
equilibrium distribution. 

We make no claim of having a rigorous mathematical 
proof of these statements. We do have an incomplete, but 
quite suggestive, analytical discussion |^], which cannot 
fit in the space available here. The latter part of this 
letter will present a few very convincing numerical simu- 
lations for symplectic maps of 2 and 4 dimensions. Ref. 
Q contains more map results, as well as a 3-dimensional 
flow. Some of our ideas are already present in ref. in- 
cluding the three-stage idea; but the crucial connection 
with the KS entropy-rate, valid for any number of dimen- 
sions and for nonlinear systems, is not there. Note that 
the definition of a single global k is not always a useful 
one, for instance for several weakly coupled subsystems; 
in such cases the connection clearly needs generalization. 

The definition of the KS entropy-rate can be found 
in many textbooks To calculate it here, we use the 
fact that it is equal to the sum of the positive Lyapunov 
exponents Our definition for the out-of-equilibrium 
physical entropy is S — constant — /, where / is the Shan- 
non information ||] . These S and / are coarse grained [|j . 
Coarse-graining consists in performing a slight smearing, 
or smoothing, of the probability distribution in phase 
space before calculating S or /. The fine-grained quanti- 
ties do not vary with time at all, because Liouville's the- 
orem says that the volume of phase space is conserved. 
The shape of that volume, however, becomes increasingly 
complicated and fractalized, due to the chaotic dynam- 
ics. Hence, under smoothing, the volume occupied keeps 
increasing. There are many ways to perform a coarse- 
graining. For this paper, we assume that phase space is 
divided into a large number of cells with volumes Va, 
such that J2a'^a = V, the total volume of available phase 
space. Then we define / by 
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where Pa {t) is the probabihty that the state of the system 
in phase space at time t faUs inside ceU Cq. . In the foUow- 
ing it wiU be more convenient to work with I{t) rather 
than S(t). This is because, when / is used, there is a con- 
venient reference point, the uniform distribution, whose 
I always vanishes, irrespective of the coarse-graining. On 
the other hand the constant quantity I+S, and therefore 
S, do depend on the coarse-graining. 

This type of coarse-graining allows an alternative ver- 
sion of the significance of k for the evolution of a physical 
system. Let us assume the initial distribution to be very 
strongly localized in phase space, i.e. most cells con- 
tain zero probability initially. Then, during the generic 
"second stage" mentioned earlier, the total number of 
occupied cells, i.e. cells with non- vanishing pa, varies 
proportionally to e"*. Our simulations verify this fact 
well (see fig. 3 later). 

We return to the need for averaging, in our simu- 
lations, many histories starting from different parts of 
phase space. This has to be done whenever the local 
K (the sum of the positive local Lyapunov exponents) 
varies appreciably from place to place, which is the nor- 
mal case for nonlinear systems. For linear maps (like the 
generalized cat map below) it is not necessary. For other 
systems, it would never be necessary if we could use a 
fine enough coarse-graining, to give the probability time 
to spread throughout phase space before any appreciable 
increase in entropy. Unfortunately such fine grain would 
require computers far more powerful than exist now. In 
the real thermodynamical world with many many dimen- 
sions, what kind of coarse-graining should preferably be 
used is, we believe, a wide open question. 

For what may be the simplest of all conservative 
chaotic systems, the baker's map, the correctness of our 
three-stage description for the behavior of I{t) can be 
shown analytically Our first simulations are done 
with the "generalized cat map" inside a unit square: 

P = p + kq (modi), 

Q=p+{l + k)q (modi) (2) 

where fc is a positive control parameter. Fig. 1 shows I(t) 
for four values of k (see caption). The coarse-graining 
grid is obtained by dividing each axis into 400 equal seg- 
ments. The initial distribution consists of 10^ points 
placed at random inside a square whose size is that of 
a coarse-graining cell, and the center of that square is 
picked at random anywhere on the map. Each of the 
four curves is an average over 100 runs, i.e. 100 histories 
with different initial distributions chosen at random, as 
mentioned. Each curve shows clearly the stage-2 linear 
behavior, the negative of the slope being accurately given 
by the (analytically calculable) Lyapunov exponent: 

A = logi(2-|-fc+ v/fc2 +4fc) = K . (3) 



Fig. 2 shows how I{t) depends on the initial distribu- 
tion and on the coarse-graining. Now k = 1 only. Other 
conditions are the same as in fig. 1, except that we cal- 
culate a single history instead of averaging 100. This 
makes no big difference in this case, because the local 
Lyapunov exponent is the same everywhere. In the six 
top curves six different sizes are compared for the initial 
square; the grid is as in fig. 1. The first size is that of 
1 coarse-graining cell, then 4 cells, 16, 64, 256, and 1024 
cells (from top to bottom) . All six curves have the same 
stage-2 slope given by —k. Their vertical displacement 
is log 2 for each factor 2 in the linear dimension of the 
initial distribution. The three bottom curves show how 
I{t) depends on the coarse-graining. The size of the ini- 
tial square is always that of 1024 original coarse-graining 
cells. For the upper curve (sixth from the top) the cells 
are as in fig. 1, for the middle curve they are squares four 
times larger in area, and for the bottom curve they are 
four times larger again. Once again, all three curves have 
the same stage-2 slope of —k. They are displaced verti- 
cally from each other by the log of the factor in coarse- 
graining linear dimension, i.e. log 2. 

In fig. 3 we plot on a log scale the number of occupied 
cells vs. time for the finest coarse-graining and 3 pro- 
gressively larger initial distributions, whose centers are 
picked at random as always. All 3 stage-2 straight lines 
are indeed fitted by e***. Their vertical displacement is 
by a factor 4, which is also the factor in the linear size of 
the initial distributions. 

The second system studied is the standard map [ p^ , 
again a two-dimensional conservative map in the unit 
square, but this time nonlinear: 

k 

P = P + TT- sm{2TTq) (mod 1) , 

Q = q + P (modi). (4) 

The map is only partially chaotic, but the percentage 
of chaos increases with the control parameter fc, and we 
use large values of k, namely 20, 10, and 5. For k = 5 
there are two sizeable regular islands, associated with a 
period 2 stable trajectory. We calculated the Lyapunov 
exponent numerically, leaving out the regular islands for 
k = 5. This yielded A = k = 2.30, 1.62, 0.98, respec- 
tively for the three fc's. Fig. 4 shows the three curves 
I{t), with the top curve corresponding to the smallest k. 
The coarse-graining grid, the choice of initial distribu- 
tion, and the averaging are the same as for fig. 1, but it 
was necessary to include 1000 histories in the averaging 
for k = 5. Each curve has a stage-2 linear portion whose 
slope is correctly given by —k. Fig. 5 presents 3 single 
histories (x's) for /c = 5, as well as the average curve 
from fig. 4 (circles). For such a very nonlinear system, 
with too coarse a grain, the single curves vary wildly and 
the averaging is essential. 

Our next example is a four-dimensional system, a 
generalized cat map. It is a linear symplectic map 
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PH , reduced to a unit-size hypercube by introducing 
"mod 1" in each of the four transformation equations. 
The two positive Lyapunov exponents Ai and A2 can 
be calculated analytically, and the KS entropy-rate is 
K — Ai -I- A2. For this system we made up the coarse- 
graining grid by dividing each of the four axes into 20 
equal segments. The initial distribution consists of 10^ 
points placed at random inside one hypercube of size 
(400)"^ X (400)"^ X (400)"^ x (400)"\ and the center 
of the hypercube is picked at random anywhere on the 
map. Fig. 6 shows I{t) for 2 single histories. They dif- 
fer greatly in their stage 1, but both have nearly linear 
stage 2's with the correct slope given by — k. 

Finally we have a 4-dimensional nonlinear map, made 
up of two coupled standard maps : 

fci 

Pi = Pi + TT- sin(27r(7i) (mod 1) 
2tt 

Qi = qi+ Pi+ mP2 (mod 1) 

^2 

P2 = P2 + TT- sin(27r(72) (mod 1) 

Q2 = q2 + P2 + mPi (modi). (5) 

We worked with 3 sets of control parameters (fci,fc2), 
namely (10,5), (5,3), and (3,1). The couphng parameter 
m was 0.5 in all cases. We calculated the two Lyapunov 
exponents numerically |^2|, then added them up to get 
K. The numerical values are in the figure caption. The 
initial volume was the size of one cell, and we averaged 
over 100 histories. Fig. 7 shows I{t). It has a fairly well 
defined second stage with a slope close to —k in all cases. 

The work we have reported makes very explicit the 
connection between the KS entropy-rate, when it is 
meaningful, and the time dependence of the physical en- 
tropy or information. Yet it is far from a complete an- 
swer. It assumes that, at the beginning of the system's 
evolution, the probability distribution spreads very fast 
to all corners of phase space before undergoing much 
fractalization, so that its subsequent variation is well 
described by a global k. But there are other possible 
scenarios. For instance there may be gradual overall evo- 
lution in space as well as in time, a possibility which is 
not included by our assumption that all parts of phase 
space must start on an equal footing. In conclusion, al- 
though this work constitutes only a quick foray into the 
subject, we hope that our assertions can function as guid- 
ing principles for research attempting to bring together 
the mathematics of chaos and the physics of far-from- 
equilibrium thermodynamics. 
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FIG. 1. Generalized Cat Map, k = 10,3,1,0.5 (from left to 
right), K = 2.48,1.57,0.96,0.69 respectively, N = lO'', grid 
= 400 x 400, Vi = Vccu, average of 100 histories. 
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FIG. 2. Generalized Cat Map, k = l,K = 0.96, N = 10®; 

from top to bottom: grid = 400 x 400, Vi/Vcdi = 1, 4, 16, 
64, 256, 1024 (six curves), and grid = 200 x 200, 100 x 100, 
Vi/Vcew = 1024 (two curves), 1 history. 
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FIG. 3. Generalized Cat Map, k= 1,k = 0.96, = 10®, 
grid = 400 x 400, Vi/Vcen = 1,16,256 (bottom to top), 1 
history. 
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FIG. 4. Standard Map, k = 20,10,5 (from left to right), 
K = 2.30, 1.62,0.98 respectively, = 10®, grid = 400 x 400, 
Vi = Keii, average of 100 hist.(fc = 20, 10), 1000 hist.(fe = 5). 




FIG. 5. Standard Map, k = 5, n — 0.98, 3 single histories 
compared with the average from fig. 4. 
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FIG. 6. 4-Dimensional Generahzed Cat Map, Ai = 0.223, 
Aa = 0.247, k = 0.470, N = 10®, grid = 20*, Vi = (20)-*Vceii, 
2 single histories. 
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FIG. 7. 4-Dimcnsional Nonlinear Map. Bottom: ki = 10, 
k2 = 5, m = 0.5, Ai = 1.65, Aa = 0.75, k = 2.40. Middle: 
fci = 5, = 3, m = 0.5, Ai = 1.03, A2 = 0.45, /t = 1.48 . 
Top: fei = 3, = 1, m = 0.5, Ai = 0.62, A2 = 0.16, k = 0.78 . 
AT = 10®, Grid = 20*, Vi = Keii, average of 100 histories. 
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